[WIP] (proof-of-concept) sqlite port of etcd #16328
Conversation
Change-Id: I2dc656d216732582693346a5d79186c73af7c5ec Signed-off-by: Han Kang <[email protected]>
Change-Id: Ied315a6089c9ee280d4e2ea33179831c29bc7c0b Signed-off-by: Han Kang <[email protected]>
Signed-off-by: Han Kang <[email protected]>
use non-cgo implementation first, so that builds continue to function Change-Id: I05aa62ad56a771552e81d397e59570737767346f Signed-off-by: Han Kang <[email protected]>
|
Normal ops do not matter that much. They are bounded by gRPC and disk performance more likely as etcd batches requests aggressively and the read path would probably not even hit disk io... Compaction and snapshot performance and isolation probably matters more. |
|
I did some memory profiling of the sqlite backend.
They both consume similar amount of memory. Even though sqlite should have much lower memory footprint in theory, there is not clear advantage of using sqlite over bbolt.
|
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions. |
|
What are the next steps for this? Is there any follow-up discussion or plans for possible adoption? Specifically, are there noticeable benefits when handling defragmentation? |
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
[DO NOT MERGE] This is just a demonstrative PR, demonstrating the proof-of-concept of:
If you want to run etcd using sqlite, please apply this patch and run:
makerm -rf default.etcd./bin/etcdYou can benchmark the performance of etcd using etcdctl by running
./bin/etcdctl check perf.Note that this implementation of etcd-on-sqlite uses a pure golang version of sqlite, which is roughly half as fast as the full blown C implementation. Even so, the etcdctl performance tests suggest that the full golang version of sqlite seems roughly on par with bolt (if not a tad bit faster). For speed, we may want to consider using
github.com/mattn/go-sqlite3, which compiles with CGO. However, this does break all of our build scripts and github actions since we presume that etcd can be compiled with CGO disabled.All github actions (unmodified) are passing.